AD-A210  532 


Jnc  ^lU  Uisri  - 


ARI  Research  Note  89-36 


Battle  Simulation  Outcomes  as  Potential 
Measures  of  BCG  Performance  in 
CATTS  Exercises 


Gary  S.  Thomas 
U.S.  Army  Research  Institute 


Field  Unit  at  Fort  Leavenworth,  Kansas 
Stanley  M.  Halpin,  Chief 


Systems  Research  Laboratory 
Robin  L,  Keesee,  Director 


June  1989 


DTIC 

Selects 
JUL211989  I  1 

.6^  ® 


United  States  Army 

Research  Institute  for  the  Behavioral  and  Social  Sciences 

Approved  for  the  public  release;  disuibution  is  unlimited. 


4 


U.S.  ARMY  RESEARCH  INSTITUTE 

FOR  THE  BEHAVIORAL  AND  SOCIAL  SCIENCES 


A  Field  Operating  Agency  Under  the  Jurisdiction 
of  the  Deputy  Chief  of  Staff  for  Personnel 


EDGAR  M.  JOHNSON 
Technical  Director 


JON  W.  BLADES 
COL,  IN 
Commanding 


Technical  review  by 
Lloyd  M.  Crumley 


NOTICES 

DISTRIBUTION;  This  report  has  been  cleared  for  release  to  the  Defense  Technical  Information 
Center  (DTIC)  to  comply  with  regulatory  requirements.  It  has  been  given  no  primary  distribution 
other  than  to  DTIC  and  will  be  available  only  throu^  DTIC  or  the  National  Technical  Informational 
Service  (NTIS). 

FINAL  DISPOSITION:  This  report  may  be  destroyed  when  it  is  no  longer  needed.  Please  do  not 
return  it  to  the  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences. 

NOTE:  The  views,  opinions,  and  findings  in  this  report  are  those  of  the  author(s)  and  should  not  to 
be  construed  as  an  official  Department  of  the  Army  position,  policy,  or  decision,  unless  so 
designated  by  other  authorized  documents. 


REPORT  DOCUMENTATION  PAGE 


REPORT  SECURITY  CLASSIFICATION 
Unclassified 


2».  SECURITY  CLASSIFICATION  AUTHORITY 


2b.  DECLASSIFICATION /OOWNCRAOING  SCHEDULE 


form  Approved 
0MB  No.  0704-0188 


1b.  RESTRICTIVE  MARKINGS 


3.  DISTRIBUTION /AVAILABILITY  OF  REPORT 

Approved  for  public  release; 
distribution  is  unlimited. 


4.  PERFORMING  ORGANIZATION  REPORT  NUMBER(S) 
ARI  Research  Note  89-36 


5.  MONITORING  ORGANIZATION  REPORT  NUM8ER(S) 


NAME  OF  PERFORMING  ORGANIZATION  6b  OFFICE  SYMBOL  7a. 
O.S.  Army  Research  Institute  (If  tppiksbie) 

for  the  Behavioral  and  Social 


7a.  NAME  OF  MONITORING  ORGANIZATION 


Sciences  I  PERI-SL 


6c  ADDRESS  (Oty,  State,  and  ZIP  Code) 

ARI  Field  Unit  at  Fort  Leavenworth,  KS 

P.O.  Box  3407 

Fort  Leavenworth,  KS  66027-0347 


8b.  OFFICE  SYMBOL 
(If  applicable) 

PERI-S 


8c  ADDRESS  rO<y,  State,  and  ZIP  Code) 
5001  Eisenhower  Avenue 
Alexandria,  VA  22333-5600 


7b.  ADDRESS  (C/fy,  State,  and  ZIP  Code) 


9.  PROCUREMENT  INSTRUMENT  IDENTIFICATION  NUMBER 


10.  SOURCE  OF  FUNDING  NUMBERS 


PROGRAM 
ELEMENT  NO. 

62785A 


1 1 .  TITLE  (Indude  Security  Classification) 

Battle  Simulation  Outcomes  as  Potential  Measures  of  BCG  Performance  in  CATTS  Exercises 


13a.  TYPE  OF  REPORT 
Final 


13b.  TIME  COVERED 
FROM  1982  TO  1983 


15.  PAGE  COUNT 

24 


16.  SUPPLEMENTARY  NOTATION  This  report  was  originally  published  in  199^  as  an  internal  ARI 
document  not  available  for  general  distribution.  It  has  been  madeXavailable  because  it 
contains  data  now  of  general  interest  to  C2  researchers.  \ 


17.  COSATI  CODES  |  18.  SUBJECT  TERMS  (Contmue  on  reverie  rf  necesM/y  an»i(  Identify  by  Wocfc  nu/nber^ 

GROUP  I  SUB'GROUP  |  Oommand  Command  and  control 

Control  \  Staff  evaluati^ 

Battle  outcome  V  Performance  ' 


19,  ABSTRACT  (Continue  on  reverse  if  necessary  and  identify  by  block  number)  len  Battalion  Command  Groups  consisting 
of  a  battalion  commander,  an  SI,  S2,  S3,  S4,  four  company  commanders,  and  roughly  ten  sup¬ 
porting  staff  members  were  exercised  on  the  CATTS  battle  simulator.  CATTS^^provides  a  con¬ 
siderable  quantity  of  battle  status  data  so  it  is  possible  to  compute  loss  exchange  ratios, 
relative  exchange  ratios,  surviving  maneuver  force  ratio  differentials,  command  and  control 
index  of  lethality  ratios,  and  change  in  combat  ratios — all  of  which  are  considered  battle 
outcome  measures  when  computed  at  the  end  of  the  battle.  Results  showed  that,  although  all 
measures  were  calculated  from  the  same  battle  data,  correlations  between  them  were  not  always 
high  (.96  to  .02)  and  were  sometimes  negative  (-.35  to  -.19)  Attempts  to  relate  performance 
measures  to  battle  outcome  measures  were  unsuccessful  and  in  the  case  of  subjective  evalua¬ 
tions  by  controllers,  there  was  reason  to  believe  that  the  data  were  contaminated  by  the 
controllers’  dual  roles  as  evaluator  and  training  event  controller.  More  research  and  higher 
quality  data  will  be  required  to  better  evaluate  the  relationship  between  battle  outcome 
measures,  and  between  those  measures  and  task  performance. 


20.  DISTRIBUTION /AVAILABILITY  OF  ABSTRACT 

(2  UNCLASSIFIEO/UNLIMITED  □  SAME  AS  RPT  □  OTIC  USERS 

21.  ABSTRACT  SECURITY  CLASSIFICATION 

Unclassified 

22a.  NAME  OF  RESPONSIBLE  INDIVIDUAL 

Lloyd  M.  Crumley 

22b.  TELEPHONE  (Include  Area  Code) 
(913)  684-4933 

22c.  OFFICE  SYMBOL 

PERI-SL 

OD  Form  1473,  JUN  86 


Previous  editions  are  obsolete. 


SECURITY  CLASSIFICATION  OF  THIS  PAGE 
UNCLASSIFIED 


i 


BATTLE  SIMULATION  OUTCOMES  AS  POTENTIAL  MEASURES  OF  BCG  PERFORMANCE  IN  CATTS 
EXERCISES 


CONTENTS _ 

Page 

INTRODUCTION  .  1 

METHOD  .  2 

Participants . 2 

CATTS  Exercises .  3 

Battle  Outcome  Components  .  3 

Subjective  Ratings  .  3 

Procedure  .  5 

RESULTS  .  6 

Controller  Ratings  of  Performance  .  6 

OPFOR  and  Friendly  Losses .  6 

Initial  Combat  Ratio  .  10 

Simulation  Outcomes  and  Performance  Ratings  .  11 

GENERAL  DISCUSSION  .  14 

REFERENCES .  16 

APPENDIX  A.  DETERMINATION  OF  INITIAL  STRENGTH  .  A-1 

B.  CALCULATIONS  FOR  SIMULATION  OUTCOMES  .  B-1 

C.  BATTLE  PERFORMANCE  INDICES  .  C-1 

LIST  OF  TABLES 

Table  1.  Weighted  levels  of  forces  and  losses  .  4 

2.  Rank  order  comparisons  of  simulation  outcomes 
and  controller  ratings  of  performance  for 

units  and  missions .  12 

3.  Correlations  between  simulation  outcomes  and 
performance  ratings  for  unit  type  combining 

across  mission  type .  12 

4.  Correlations  between  simulation  outcomes  and 
performance  ratings  for  mission  type  combining 

across  unit  type  and  for  all  exercises  combined .  13 

ili 


Page 


Table  5. 


Figure  1 


2 

3 


LIST  OF  TABLES  (Continued) 

Intercorrelations  between  candidate 

measures  of  battlefield  performance  .  13 


LIST  OF  FIGURES 


Main  controller  performance  ratings  for 
Hech,  Cav,  and  Inf  units  conducting  defense 

and  attack  missions  .  7 

Mean  losses  received  by  friendly  units  for 

defensive  and  attack  missions  .  7 

Mean  OPFOR  losses  inflicted  by  friendly 

units  for  defensive  and  attack  missions  .  9 

Mean  initial  combat  ratio  for  Mech,  Cav,  and 
Inf  units  conducting  defensive  and  attack 

missions  .  9 


^  / 


Accession  For 

NTIS  GRA&I 

DTIC  TAB 
Unannounced 
Justlf loot lon_ 


By - 

Distribution/ 


Availability  Codes 
Avail  and/or 
Special 


iv 


□  □ 


Battle  Sinmlatlon  Outcomes  as  Potential  Measures 
of  BCG  Performance  in  CATTS  Exercises 


In  the  past  several  years  battle  simulations  have  gained  Increased 
credibility  as  systems  for  training  command  and  control  (C^)  processes 
at  tactical  echelons  from  battalion  through  corps.  Of  the  battle  simula¬ 
tions  developed  to  support  training,  the  most  sophisticated  In  terms 
of  the  extent  of  automation  and,  therefore,  ability  to  represent  battle¬ 
field  events  in  real  time,  is  the  Combined  Arms  Tactical  Training  Simulator 
(CATTS).  CATTS  is  used  to  train  battalion  command  groups  (BCG)  and 
serves  as  a  test  bed  to  improve  training  procedures  and  to  specify 
requirements  for  future  sl^latlons.  In  order  to  address  training 
Issues  within  the  context  of  the  CATTS  system.  It  Is  necessary  first  to 
develop  measures  of  BCG  performance,  so  that  changes  In  C^  performance 
can  be  determined  and  used  to  assess  the  training  value  of  various  CATTS 
characteristics  and  procedures. 

During  the  last  several *y^ars,  API  has  conducted  research  directed 
toward  developing  a  variety  of  BCG  performance  measures  In  CATTS  exer¬ 
cises.  These  efforts  have  resulted  in  a  questionnaire  based  on  a  modified 
form  of  the  Army  Training  and  Evaluation  Program  (ARTEP)  (Barber  and 
Kaplan,  1979;  Kaplan  and  Barber,  1979)  that  has  been  used  to  assess  the 
extent  to  which  ARTEP  subtasks  are  exercised  In  CATTS.  The  BCG  players 
participating  In  CATTS  exercises  and  the  (^TTS  control  staff  have  rated 
player  performance  on  ARTEP  tasks  and  subtasks  using  this  questionnaire. 
Also  developed  by  ARI  and  the  Automated  Command  Training  Division  (A(JTD) 

Is  an  Information  flow  questionnaire  designed  to  determine  which  players 
received  and  transmitted  Information  required  by  the  group  to  effectively 
plan  and  execute  a  mission  during  CATTS  exercises  (Kaplan,  1980) .  These 
Instruments  were  designed  primarily  to  assess  the  process  by  which  BCGs 
coordinate  and  implement  a  battle  plan,  and  do  not  directly  assess  the 
consequences  of  this  process.  The  following  discussion  considers  the 
usefulness  of  simulated  battle  outcomes  as  potential  measures  of  the 
consequences  of  these  BCG  processes. 

Simulation  outcomes  could  complement  the  performance  measures  already 
being  collected  and  could  be  especially  valuable  due  to  their  less 
subjective  nature.  There  are,  however,  several  questions  associated 
with  using  battle  outcomes  as  performance  measures  in  CATTS  that  have 
prevented  their  utilization  in  the  past.  First,  to  what  degree  are  the 
outcomes  of  simulated  battle  engagements  realistic;  and  second,  to  what 
degree  are  the  outcomes  a  valid  Indication  of  BCG  performance?  The 
validity  question  points  to  the  need  for  some  criterion  measure  of  BCG 
performance  that  simulation  outcomes  can  be  validated  against,  or  for  a 
procedure  for  attributing  validity  to  the  battle  outcomes. 
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The  detenainistlc  nature  of  the  simulation  outcomes  can  be  addressed 
in  terms  of  the  CATTS  software  and  Internal  algorithms.  Such  variables 
as  weapons  effects,  priority  of  fire,  movement  rates,  and  loss  probabil¬ 
ities  are  all  programmed  into  the  CATTS  system.  The  computer  takes  from 
CATTS  controllers,  input  based  on  C^  decisions  by  the  BCG,  and  determines 
the  battlefield  outcomes  as  a  result  of  those  Inputs.  The  battle  simula¬ 
tion  outcomes  are  largely  deterministic  rather  than  probabllstic,  since 
replications  of  the  same  maneuvers  and  engagements  result  in  very 
similar  friendly  and  OPFOR  losses. 

TRASANA  (1979)  has  investigated  the  battle  calculus,  movement  routines, 
line  of  sight  calculations,  weapons  effects  curves,  etc.,  of  CATTS  and 
has  deteinnlned  them  to  be  realistic  representations  of  actual  battle. 

•In  addition,  continuous  efforts  have  been  made  by  ACTD  over  the  last 
several  years  to  further  refine  the  CATTS  algorithms,  so  that  more 
realistic  outputs  are  derived  from  the  inputs  to  the  system. 

Currently  there  is  no  absolute  criterion  measure  of  BCG  performance 
available  to  which  the  simulation  outcomes  from  CATTS  can  be  compared. 

But,  in  terms  of  validity,  the  outcomes  of  battle  should:  (1)  reflect 
expected  differences  in  battle  outcomes  for  different  types  of  missions 
(e.g.,  covering  force  vs.  attack);  (2)  reflect  expected  outcomes  due  to 
initial  combat  ratio  (relative  combat  power)  of  OPFOR  vs.  friendly 
forces;  and  (3)  reflect  a  systematic  relationship  to  subjective  ratings 
of  BCG  performance  by  CATTS  controllers.  The  current  paper  is  an  initial 
examination  of  the  validity  attributable  to  simulation  outcomes  as 
measures  of  BCG  performance. 


METHOD 

Participants 

Participants  were  ten  BCGs  consisting  of  a  battalion  commander,  an 
SI,  S2,  S3,  SA,  four  company  commanders,  and  roughly  ten  supporting 
staff  members.  The  BCGs  exercised  in  CATTS  between  October  1980  and 
October  1981.  BCGs  were  either  mechanized  Infantry  (n>A) ,  armored 
cavalry  (n*3) ,  or  light  Infantry  (n“3),  that  conducted,  on  either  Fulda 
or  Sinai  terrain,  a  covering  force  (mech  and  cav)  operation  or  a  static 
defense  (infantry  only),  followed  by  an  attack  on  the  next  day.  All 
mechanized  and  cavalry  units  and  one  Infantry  unit  exercised  on  the 
Fulda  Gap  terrain.  However,  the  terrain  was  slightly  different  for 
each  unit  type.  The  remaining  two  infantry  units  exercised  on  a  Sinai 
terrain.  The  OPFOR  typically  deployed  in  basically  the  same  fashion  in 
all  equivalent  missions  (attack  or  defense).  BCGs  conducted  missions  in 
conjunction  with  adjacent  battalions  controlled  by  a  brigade  staff, 
played  by  CATTS  controllers. 
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CATTS  Exercises 


On  the  first  day  of  CATTS  exercises,  BCG  staffs  received  the  brigade 
operations  order  from  CATTS  controllers  playing  the  roles  of  brigade  SI, 

S2,  S3,  and  S4.  While  the  BCG  staffs  were  planning  their  operations 
orders,  CATTS  controllers,  functioning  as  maneuver  controllers,  fire 
support  officers,  and  computer  Interactors  trained  BCG  company  comntananders 
and  their  fire  support  personnel  to  participate  In  CATTS  exercises.  On 
day  two,  BCG  players  conducted  a  defensive  mission  (covering  force  or 
static  defense).  At-  the  conclusion  of  the  exercise,  CATTS  controllers 
rated  the  performance  of  the  BCGs.  On  the  third  day,  BCGs  received  a 
frag  order,  conducted  an  attack,  and  were  again  rated  by  CATTS  controllers. 

Battle  Outcome  Components 

Computer-generated  summary  reports  of  battle  status  of  opposing 
forces  provided  primary  data  for  this  Investigation.  These  reports 
Indicated  initial  level  of  equipment  In  all  OPFOR  and  friendly  units, 
plus  a  summary  of  those  pieces  of  equipment  lost  by  opposing  units 
through  the  course  of  the  battles.  CATTS  battle  calculus  Included  a 
relative  weight  for  each  weapon  system  according  to  Its  degree  of 
fire  power.  For  example,  an  M60A1  tank  had  a  weight  of  73,  whereas  an 
M113  APC  had  a  weight  of  19.  The  weighting  system  contributed  to  the 
computer  decision  rules  that  determined  which  types  of  equipment  fired 
at  and  destroyed  other  equipment  types.  This  weighting  system  was  used 
in  the  current  research  to  determine  relative  initial  strength  and  the 
relative  battle  losses  of  OPFOR  and  friendly  forces.  This  was  accomplished 
by  multiplying  the  weighting  factors  by  the  number  of  the  corresponding 
equipment  types  under  consideration.  The  products  were  then  summed 
across  all  OPFOR  units  and  friendly  forces,  respectively,  that  were 
present  on  the  battlefield.  The  total  weighted.  Initial  strength  of 
OPFOR  and  friendly  forces  appear  in  columns  one  and  two  of  Table  1.  The 
table  also  Indicates  the  initial  combat  ratio  (OPFOR  initial  forces 
divided  by  initial  friendly  forces)  and  the  total  losses  sustained  by 
opposing  forces  in  the  simulated  battles.  Appendix  A  includes  the 
rationale  for  determining  initial  force  levels. 

Subjective  Ratings 

The  other  source  of  data  was  CATTS  controller  ratings  of  overall  BCG 
performance  on  each  exercise  day.  The  number  of  controllers  who  rated 
BCG  performance  in  each  CATTS  exercise  ranged  from  five  to  nine  depending 
upon  the  availability  of  experienced  controllers.  Since  the  rating 
Instrument  used  a  magnitude  estimation  technique  (Stevens,  1973),  raw 
ratings  were  converted  to  log^^g  to  normalize  the  distribution  of  scores. 
Those  logs  were  further  converted  to  Z-scores  for  each  individual  rater 
in  order  to  control  for  systematic  rater-response  bias.  A  mean  standard¬ 
ized  performance  score  was  then  computed  for  each  BCG  on  each  exercise 
day. 
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Procedure 


The  battle  outcome  components  compiled  In  Table  1  for  all  BCGs 
participating  in  defensive  and  attack  missions  served  as  data  to  compute 
the  simulation  outcomes  as  potential  measures  of  battlefield  perform 
mance.  These  Include  loss  exchange  ratio  (LER) ,  a  relative  exchange 
ratio  (R£R),  and  surviving  maneuver  force  ratio  differential  (SMFRD) , 
which  have  all  been  used  in  previous  research  and/or  combat  modeling 
(USACDC,  1973).  All  these  measures  are  mathematical  relationships  of 
OPFOR  and  friendly  losses  or  surviving  forces  at  the  end  of  an  entire 
battle.  The  mathematical  formulas  used  to  compute  these  measures  appear 
in  Appendix  B.  The  LER  is  simply  computed  by  dividing  the  weighted 
total  OPFOR  loss  by  friendly  weighted  total  losses;  RER  is  a  ratio  of 
the  percentage  of  OPFOR  losses  to  the  percentage  of  friendly  losses;  and 
SMFRD  is  the  percent  of  friendly  forces  surviving  minus  the  percent  of 
OPFOR  surviving  battle.  Two  other  potential  measures  of  battle  perfor¬ 
mance  were  generated  to  predict  overall  controller  ratings  of  BCG 
performance:  the  Command  and  Control  Index  of  Lethality  Levels  (CTILL) 
ratio  and  the  change  in  combat  ratio  (ACR) .  Calculations  of  these  also 
appear  in  Appendix  B. 

All  other  things  being  equal,  it  is  preferable  to  have  a  higher 
proportion  of  friendly  forces  surviving  battle  and  to  destroy  a  high 
proportion  of  enemy  weapon  systems.  This  assumption  lead  to  the  form¬ 
ulation  of  the  C^ILL  ratlOi  which  was  calculated  by  adding  the  percentage 
of  OPFOR  losses  to  one-half  the  percentage  of  friendly  forces  surviving. 
Preliminary  data  analysis  indicated  that  in  assessing  player  perfor¬ 
mance,  the  controllers  apparently  were  placing  more  emphasis  on  OPFOR 
attrition  than  on  the  proportion  of  friendly  forces  lost  or  surviving. 
Therefore,  in  the  current  formula,  percent  of  friendly  forces  was 
divided  in  half,  so  that  it  deemphasized  friendly  surviving  with  respect 
to  OPFOR  losses.  It  may  be  noted  that  the  formulas  for  C^ILL  and  SMFRD 
are  similar  with  the  exception  of  the  above  weighting  factor  applied  to 
friendly  forces  surviving. 

The  other  potential  measure  of  battlefield  performance  (ACR) ,  was 
formulated  on  the  assumptlop  that  it  is  preferable  to  end  the  battle 
with  a  more  advantageous  combat  ratio  than  existing  prior  to  battle.  As 
indicated  in  Table  1,  OPFOR  typically  had  a  combat  power  advantage  prior 
to  battle  as  depicted  by  initial  combat  ratios  in  excess  of  1.0.  The 
degree  to  which  the  friendly  forces  can  change  that  combat  ratio  during 
battle  to  a  more  favorable  ratio,  should  be  an  Indication  of  how  well 
they  employed  weapon  systems  against  the  OPFOR.  This  change  in  combat 
ratio  (a'CR)  was  calculated  by  subtracting  the  end  of  battle  combat  ratio 
from  the  initial  combat  ratio  and  dividing  the  difference  by  the  initial 
combat  ratio.  Higher  positive  ACR's  indicate  better  battlefield  per¬ 
formance.  Higher  positive  values  are  also  preferable  on  the  other 
measures  of  battlefield  performance.  The  purpose  of  formulating  these 
last  post— hoc  measures  was  to  find  additional  measures  that  correlated 
with  controller  ratings,  and  that  generalized  across  more  mission  and 
unit  types. 
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RESULTS 


The  present  research,  which  was  conducted  in  a  very  uncontrolled 
environment,  attempts  to  formulate  battle  simulation  outcomes  that  bear 
some  relationship  to  controller  ratings  of  performance.  Several 
preliminary  analyses  are  presented  to  arrive  at  a  better  understanding 
of  the  components  of  these  simulation  outcomes,  particularly  losses  and 
initial  strengths  of  opposing  forces.  These  analyses  are  of  Interest 
since  the  values  of  these  components  determine  the  magnitude  of  the 
simulation  outcomes;  and  consequently,  the  relationship  between  outcomes 
and  controller  ratings.  The  following  discussion  also  identifies  some 
uncontrolled  variables  (e.g.,  initial  combat  ratio)  that  could  affect 
simulation  outcomes  and  the  relationship  between  outcomes  and  ratings. 

As  stated  previously,  battle  outcomes  (e.g.,  losses)  should  logically 
be  affected  by  such  variables  as  mission  type  and  initial  combat  ratios. 
But,  for  purposes  of  performance  evaluation  and  training,  it  is  desirable 
to  develop  measures  of  battlefield  performance  that  generalize  across  such 
variables.  The  first  analysis  investigates  the  Impact  of  some  of  these 
variables  on  controller  ratings  of  performance,  so  that  results  can  then 
be  compared  to  similar  analyses  performed  on  battle  outcomes. 

Controller  Ratings  Of  Performance 

The  controller  performance  ratings  (Appendix  C)  were  analyzed  in  a 
2X3  ANOVA  to  determine  if  the  ratings  varied  as  a  function  of  two  types  of 
mission  or  three  types  of  unit.  The  mean  values  of  performance  ratings 
for  mission  and  unit  type  are  presented  in  Figure  1.  Although  ratings 
were  higher  for  Cav  and  Inf  than  for  Mech,  the  results  of  the  ANOVA 
only  approached  significance  (F  2,7*  3.953,  P  <.10).  There  was  no 
difference  in  ratings  for  mission,  however,  the  unit  by  mission  inter¬ 
action  was  significant  (F  1,7*  7.169,  P  ^*05).  This  interaction 
apparently  reflected  the  fact  that  performance  ratings  Increased  from 
defense  to  attack  for  Mech  units  only,  while  both  Inf  and  Cav  units’ 
performance  ratings  declined  somewhat  from  defensive  to  attack  missions. 

Of  particular  interest  is  the  fact  that  Mech  units  were  rated  except¬ 
ionally  low  in  defensive  missions.  This  will  be  the  topic  of  later 
discussion. 

OPFOR  And  Friendly  Losses 

Since  controller  ratings  of  BCG  performance  were  Influenced  by  the 
mission  and  unit  type  variables,  it  is  possible  that  OPFOR  and  friendly 
losses  in  CATTS  exercises  were  also  affected  by  these  variables.  The 
potential  effects  are  of  Interest,  since  equipment  losses  are  the  major 
components  of  the  simulation  outcome  measures  to  be  discussed  later. 
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Performance  Ratings  in  Z  Scores 


All  else  being  equal.  It  can  be  argued  that  units  which  attrit  large 
numbers  of  enemy  equipment  while  minimizing  losses,  should  typically  be 
perceived  as  performing  better  than  battalions  that  do  not.  The  follow¬ 
ing  set  of  analyses  were  conducted  to  determine  the  Influence  of  mission 
and  unit  type  on  friendly  and  OPFOR  losses.  The  mean  values  for  total 
losses  Incurred  by  friendly  units  In  the  two  mission  types  are  presented 
In  Figure  2.  An  analysis  (2X3  ANOVA)  of  friendly  losses  as  a  function  of 
mission  and  unit  type  revealed  significant  main  effects  for  both  variables, 
(Fi^7-  11.356,  P  ^  .025  and  F2,7  ■  5.836,  P  <  .05),  respectively. 

Fewer  friendly  losses  were  sustained  In  attack  missions  and  by  Mech 
units.  The  mission  effects  were  reasonable,  since  friendly  units 
t3rplcally  encounter  far  less  OPFOR  in  attack  scenarios.  The  fact  that 
Mech  units  sustained  less  casualties  and  received  lower  ratings  than  Inf 
and  Cav  units  appears  Inconsistent  with  the  above  hypothesis.  To  be 
consistent  with  the  hypothesis  and  the  lower  performance  ratings,  Mech 
units  would  have  had  to  Inflict  considerably  fewer  losses  on  the  OPFOR 
to  compensate  for  the  low  level  of  losses  they  sustained. 

Analysis  of  OPFOR  losses  (2X3  ANOVA)  does  not  resolve  this  Incon¬ 
sistency,  since  no  significant  differences  for  unit  type  were  observed. 

The  main  effect  for  mission  type  was,  however,  significant  (Fi^7  ■ 

54. A4',  P<  .005),  where  more  OPFOR  equipment  was  destroyed  In  the  defensive 
missions  than  In  attacks.  In  an  attempt  to  explain  the  lack  of  expected 
differences  In  OPFOR  losses  Inflicted  by  Mech  units  In  comparison  with 
Cav  and  Inf  units, It  was  observed  that  occasionally  during  Mech  covering 
force  missions,  excessive  air  missions  had  to  be  Initiated  by  friendly 
brigade  controllers  to  prevent  friendly  battalions  from  being  overrun  by 
OPFOR.  Since  controllers  appeared  to  be  sensitive  to  these  situations, 
and  adjusted  their  ratings  accordingly,  the  friendly  Mech  data  were 
"adjusted"  in  subsequent  analyses.  Units  whose  defensive  air  "kills" 

(Mech  5  and  Mech  10)  exceeded  those  attained  by  all  other  units  by  one 
standard  deviation  from  the  mean  for  all  units  (Mech,  Cav,  and  Inf),  had 
that  amount  of  "kills"  subtracted  from  scores  of  OPFOR  losses  Induced. 

These  "adjusted"  values  for  OPFOR  losses  Inflicted  by  friendly  units 
are  presented  In  Figure  3. 

» 

A  subsequent  reanalysis  (2X3  ANOVA)  of  OPFOR  losses  reveal  a  signi¬ 
ficant  main  effect  for  unit  type  (F2  7  *  4.985,  P<  .05),  where  Cav  and 
Inf  units  achieved  more  OPFOR  "kills"  than  did  Mech  units.  The  "stat¬ 
istical  adjustment"  described  above  results  in  findings  that  are 
consistent  with  controller  ratings  of  performance,  since  one  of  the 
main  objectives  of  defensive  operations  is  to  attrit  the  enemy.  It  is 
also  possible  that  poor  deployment  of  existing  assets  contributed  to 
both  low  OPFOR  attrition  and  low  friendly  casualties  sustained  by  Mech 
units,  especially  in  the  covering  force  mission. 
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Initial  Combat  Ratio 


Since  the  factors  unit  and  mission  type  apparently  affect  both 
performance  ratings  and  the  outcomes  of  battle.  It  Is  desirable  to 
Identify  underlying  variables  which  may  Impact  on  both  of  these  factors. 

One  such  variable  Is  the  combat  ratio  that  exists  prior  to  battle..  It 
Is  reasonable  to  assume  that  units  with  more  Initial  strength  relative 
to  the  OPFOR  are  likely  to  perform  better  on  the  battlefield;  therefore, 
analyses  were  conducted  on  Initial  combat  ratio  and  Its  relationship  to 
unit  and  mission  type.  The  mean  values  tested  for  significance  re¬ 
present  the  proportion  of  OPFOR  to  friendly  forces  at  the  beginning  of 
battle  and  are  presented  In  Figure  4.  Results  of  the  analyses  (2X3 
ANOVA)  Indicated  a  reliable  difference  for  unit  type  (F2,7  “  6.047, 

P  ^  .05),  where  combat  ratio  favoring  OPFOR  tended  to  be  higher  for 
Mech  than  for  Cav  and  Inf.  Not  surprisingly,  mission  type  was  also  a 
significant  variable  (Fi,7  ■  27.627,  P<  .005),  where  combat  ratios 
favoring  OPFOR  were  higher  In  defensive  missions.^ 

Mech  units  tended  to  be  rated  lower  by  controllers  and  also  to  be 
exposed  to  higher  and  less  favorable  combat  ratios.  In  fact,  a  cor¬ 
relation  of-.67  (P<.001),  Indicated  that  as  combat  ratio  Increased 
(became  less  favorable  to  friendly  forces) ,  controller  ratings  of  performance 
tended  to  decrease  regardless  of  mission  or  unit  type.  Battle  outcomes 
(losses)  were  also  likely  affected  by  prebattle  combat  ratio;  for  example, 
Mech  units  sustained  and  inflicted  fewer  losses  and  were  exposed  to 
poorer  Initial  combat  ratios.  Eventhough  losses  may  be  related  to 
controller  ratings  of  performance.  It  Is  apparent  that  other  factors 
contributed  to  these  ratings.  For  example,  battle  losses  and  initial 
combat  ratios  both  varied  as  a  function  of  mission  type,  but  mission 
type  did  not  significantly  influence  controller  ratings.  It  Is,  there¬ 
fore,  possible  that  controllers  adjusted  their  performance  ratings  to 
take  Into  account  mission  differences,  such  as  the  fact  that  the 
defenders  had  a  tactical  advantage  and  could  better  fight  outnumbered. 

In  addition,  the  way  in  which  BCGs  deployed  their  assets  should  In  great 
degree  determine  how  well  they  performed  on  the  simulated  battlefield. 

As  predicted.  It  Is  apparent  that  the  outcomes  of  (e.g.,  losses)  are 
affected  by  such  variables  as  unit  and  mission  type,  which  are  confounded 
with  Initial  combat  ratios.  There  is  also  some  degree  of  relationship 
between  these  variables  and  controllers  ratings  of  performance.  Since 
It  is  desirable  to  assess  simulated  battle  performance  across  these 
variables,  the  following  discussion  emphasizes  potential  ways  of 
combining  battlefield  losses  Into  composite  scores  (simulation  outcomes) 
in  order  to  predict  performance  ratings. 


As  Indicated  In  Appendix  A,  combat  ratio  as  a  function  of  mission 
type  does  not  totally  reflect  doctrine  since  the  figures  only  reflect 
combat  power  available  on  the  battlefield  and  not  the  way  in  which  it 
was  employed. 
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simulation  Outcomes  And  Performance  Ratings 


Investigations  Into  the  relationship  between  potential  measures  of 
BCG  performance  were  conducted  as  a  function  of  friendly  unit  and 
mission  type.  The  potential  measures  of  simulated  battle  performance 
were  correlated  with  controller  ratings  of  performance  (Appendix  C)  to 
arrive  at  a  direct  comparison  of  the  relationships  among  these  variables, 
as  shown  In  Tables  2  through  4. 

Table  2  Is  a  matrix  that  Indicates  the  situations  In  which  the  rank 
order  of  controller  ratings  of  BCG  performance  was  Identical  to  the  rank 
order  of  simulation  outcomes  for  each  unit  type  within  each  mission 
type.  Rank  order  of  these  variables  were  used  since  n-slze  In  each 
situation  was  too  small  to  calculate  any  meaningful  correlations.  As 
Indicated  In  the  table,  controller  ratings  are  predicted  by  the  rank 
order  of  simulation  outcomes  In  25  of  30  possible  comparisons.  Only  the 
C^ILL  ratio  rankings  perfectly  predicted  the  rank  of  controller  ratings 
In  all  six  unit  by  mission  situations. 

Table  3  represents  a  correlation  matrix  where  the  five  candidate 
measures  of  battle  performance  were  compared  to  controller  ratings  of 
performance  for  each  type  of  unit  (Mech,  Inf,  and  Cav),  calculated 
across  mission  type.  As  Indicated  by  the  table,  each  candidate  measure 
correlates  significantly  with  performance  ratings  for  one  or  more  of  the 
unit  t3rpe8,  but  not  for  all  three.  It  appears  that  the  degree  of 
relationship  Is  unit  dependent. 

Table  4  Indicates  those  battle  performance  measures  that  correlate 
significantly  with  controller  ratings  across  unit  type  when  mission 
(defensive  vs.  attack)  Is  considered  separately.  Significant  correl¬ 
ations  appear  In  the  table  for  all  comparisons  except  ACR  In  defensive 
missions  and  LER  In  both  defensive  and  attack  missions.  RER  and  C^ILL 
appear  to  be  particularly  good  predictors  when  mission  Is  held  constant. 

The  other  correlations  appearing  In  Table  4  are  between  controller 
ratings  of  performance  and  the  five  simulation  outcome  measures  for  all 
19  exercises  regardless  of  mission  or  unit  type.  As  Indicated  by  the 
table,  RER,  SMFRD,  and  C^ILL  correlate  significantly  with  controller 
ratings.  The  highest  correlations  obtained  were  for  the  C^ILL  and 
SMFRD.  These  separately  account  for  nearly  50%  of  the  variance  In 
controller  ratings,  and  combined  in  a  multivariate  analysis,  account 
for  66%  of  the  variance.  Apparently  the  factors  Including  unit  and 
mission  type  still  Impact  on  these  measures,  since  the  magnitude  of 
these  correlations  Is  not  as  high  as  those  derived  when  mission  and 
unit  tjrpe  are  controlled.  There  is  also  no  simulation  outcome  that 
appears  to  be  the  "best"  predictor  of  controller  ratings  In  all 
situations,  although  ILL  appears  to  be  the  most  conslstant. 


Table  2 


Rank  Order  Comparisons  of  Simulation  Outcomes  and 
Controller  Ratings  of  Performance  for  Units  and  Missions 

LER  RER  SMFRD  ACR  C^ILL 


Mech  covering  force  (n"A) 
Cav  covering  force  (n-3) 
Inf  defense  (n*3) 

Mech  attack  (n«4) 

Cav  attack  (n-3) 

Inf  attack  (n-2)* 


XXX 
XXX 
X  X 

X  X 

X  X 

X  X 


X 

X 

X 

X 

X 


X 

X 

X 

X 

X 

X 


X  —Identifies  situations  where  the  rank  order  of  simulation  outcomes  were 
Identical  to  the  rank  order  of  controller  ratings  of  BCG  performance. 

*  -Indicates  that  one  Infantry  unit  In  the  attack  was  excluded  from  this  and 
all  subsequent  analyses  due  to  the  fact  that  the  unit  failed  In  its  mission. 
Only  this  unit  totally  failed  In  Its  mission  resulting  in  extraordinary  low 
controller  ratings,  but  relatively  high  simulation  outcome  scores. 
Controllers  were  apparently  responding  to  the  fact  that  mistakes  made  by 
the  unit  required  adjacent  units  to  complete  the  mission. 


Table  3 


Correlations  Between  Simulation  Outcomes  and  Performance  Ratings 
for  Unit  Type  Combining  Across  Mission  Type 


LER 

RER 

SMFRD 

ACR 

C^ILL 

Mech  (n-8) 

r— .027 

r-.317 

r=.831^ 

r-.956‘= 

r-.511 

Cav  (n-6) 

r-  .812^ 

r-.918C 

r=.139 

r— .059 

r-.949‘^ 

Inf  (n-5) 

r--.420 

r-.874® 

r-.429 

r».476 

r-.820® 

®  Pearson 

correlation  with  P 

<.05 

^  Pearson 

correlation  with  P 

<.005 

^  Pearson 

correlation  with  P 

<.001 
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Table  4 


Correlations  Between  Simulation  Outcomes  and  Performance 
Ratings  for  Mission  Type  Combining  Across  Unit  Type 
and  for  All  Exercises  Combined 


LER  RER  SMFRD  ACR  c2lLL 


Defensive  missions  (n>10) 

r— .009 

r-.946C 

r-.SOO** 

r-.200 

^942*^ 

t 

Attack. missions  (n«9) 

r«  .466 

r-.901‘^ 

r-.827^ 

r-.764® 

r».953C 

All  missions  and 
unit  types  (n“19) 

r— .079 

r-.590^ 

r«.696‘^ 

r-.386 

^.697*^ 

^  Pearson  correlation  with  P  <.01 
Pearson  correlation  with  P  <.005 
c  Pearson  correlation  with  P  <.001 


Table  5 

Intercorrelations  Between  Candidate  Measures 
of  Battlefield  Performance 


LER 

RER 

SMFRD 

ACR 

C^ILL 

LER 

- 

r-.547^ 

r— .192 

r— .347 

r-.475® 

RER 

- 

.  r=  .372 

r-  .021 

r-.962C 

SMFRD 

- 

r-  .764*^ 

r-.471® 

ACR 

- 

r«.094 

C^ILL 

- 

®  Pearson  correlation  with  P  <.05 
Pearson  correlation  with  P  <.005 
^  Pearson  correlation  with  P  <.001 
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Finally,  Intercorrelations  between  the  five  candidate  measures  of 
bb&tlefleld  performance,  regardless  of  mission  or  unit  type,  are  presented 
In  Table  5.  As  Indicated  by  the  table,  there  are  several  Instances  where 
the  measures  are  related,  byt  only  RER  and  Cr  ILL  are  nearly  perfectly 
correlated.  It  may  be,  therefore,  that  these  measures  are  In  general 
assessing  somewhat  different  aspects  of  battlefield  outcomes. 


PENERAL  DISCUSSION 

This  Initial  attempt  at  Identifying  simulated  battle  outcomes  as 
measures  of  BCG  performance  appeared  to  be  successful.  The  procedure  of 
assigning  fire  power  weights,  which  are  part  of  the  CATTS  battle  calculus, 
to  Individual  weapons  systems  allowed  calculations  of  total  pre-battle 
combat  power  and  of  total  losses  through  the  course  of  simulated  battle. 
The  battle  losses  were  responsive  to  both  mission  and  unit  type  variables. 
Where  few  units  are  actually  engaged,  attack  missions  resulted  In  sign¬ 
ificantly  fewer  losses  for  both  OPFOR  and  friendly  units.  Mech  units 
also  Inflicted  fewer  casualties  on  OPFOR  than  did  Cav  and  Inf  units. 

This  may  have  been  due  to  the  fact  that  Mech  units  had  less  fire  power 
and  faced  significantly  less  favorable  Initial  combat  ratios.  These 
results  are  consistent  with  expectations  In  real  combat.  The  Inability 
of  Mech  units  to  Inflict  high  casualties  on  the  OPFOR  In  the  covering 
force  mission  appears  to  be  reflected  In  exceptionally  low  controller 
ratings  for  this  condition.  It  should  be  pointed  out  that  across  all 
defensive  missions,  the  correlation  between  performance  ratings  and 
absolute  number  of  OPFOR  losses  were  r-.96.  However,  a  comparable 
correlation  for  attack  missions  was  only  r-.08,  indicating  that  dif¬ 
ferent  factors  relate  to  controller  ratings  depending  upon  mission  type. 


The  simulation  outcomes,  which  are  composites  of  friendly  and  OPFOR 
losses,  were  typically  good  predictors  of  controller  ratings  within  some 
mission  and  unit  type  constraints.  The  LER  only  correlated  with  ratings 
In  some  cases  where  mission  and/or  unit  type  was  held  constant.  On  the 
other  hand,  RER,  SMFRD,  and  C^ILL  correlated  very  highly  with  ratings 
across  unit  type  for  both  attack  and  defensive  missions.  And,  when  the 
data  was  combined  across  all  conditions,  SMFRD  and  C  ILL  both  predicted 
overall  performance  ratings  to  a  high  degree  (r“.70). 


These  results  are  highly  promising  with  respect  to  the  potential  use 
of  simulation  outcomes  as  measures  of  BCG  performance  on  the  simulated 
battlefield  In  CATTS.  However,  In  the  current  Investigation  the  simula¬ 
tion  outcomes  were  calculated  to  maximize  the  relationship  with  perfor¬ 
mance  ratings.  Therefore,  a  shrinkage  of  the  observed  relationships 
could  be  expected  in  replications  of  the  research.  On  the  other  hand, 
the  correlations  could  improve  If  factors  such  as  initial  combat  ratio 
were  controlled  or  systematically  manipulated. 
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Another  factor  Impacting  the  battle  outcomes  Is  the  Influence  of 
CATTS  controllers  performing  the  roles  of  brigade  staff  members. 
Situations  where  their  Influence  could  be  observed  were  in  brigade 
Initiated  alrstrlkes.  In  artillery  support,  and  in  assistance  given  to 
battalion  by  adjacent  units.  It  Is  desirable  that  these  factors  be 
controlled  in  future  research  to  separate  the  influence  of  CATTS 
controllers  playing  brigade  staff  from  the  performance  of  BCG  on  the 
battlefield. 

Variables  such  as  terrain  and  factors  other  than  losses  which 
contribute  to  overall  mission  accomplishment  could  not  be  systematically 
studied  In  the  present  descriptive  Investigation.  No  highly  significant 
correlations  between  ratings  and  simulation  outcomes  were  observed  for 
Inf  units.  This  may  have  been  due  in  part  to  the  fact  that  Inf  units 
exercised  on  both  Sinai  and  Fulda  terrain.  And,  exclusion  of  data  from 
one  Inf  attack  did  result  In  higher  correlations  between  variables. 

This  data  point  was  excluded  because  the  unit  was  not  successful  In  Its 
mission,  suggesting  that  other  factors  contributing  to  the  degree  of 
mission  accomplishment  should  be  Identified  and  Included  with  simulation 
>  outcomes  as  measures  of  BCG  performance. 

Finally,  there  Is  the  Issue  of  no  mutually  agreed  upon  criterion 
measure  of  BCG  performance.  Research  should  be  directed  at  Identifying 
such  a  measure,  so  that  the  relationship  of  measures  such  as  performance 
ratings.  Intrastaff  Information  flow,  and  battlefield  outcomes  to 
overall  BCG  performance  can  be  assessed.  VJhen  these  relationships  are 
determined.  It  would  be  possible  to  develop  an  additional  set  of 
performance  measures,  which  could  be  used  as  part  of  a  diagnostic/ 
feedback  package  for  BCGs.  This  feedback  could  support  the  training 
needs  of  BCGs  In  computer-driven  command  and  control  training  exercises 
by  providing  Indications  of  BCG  strengths  and  weaknesses. 

In  conclusion,  some  degree  of  validity  can  be  attributed  to  simula¬ 
tion  outcomes  as  measures  of  BCG  performance  In  CATTS  exercises.  The 
components  of  those  formulas,  battlefield  losses,  were  sensitive  to 
changes  In  mission  and  unit  type,  and  probably  differences  in  pre-battle 
combat  ratios.  Results  are  particularly  promising  due  to  high  correl¬ 
ations  between  simulation  outcomes  and  controller  ratings  of  BCG 
performance.  Further  research  on  the  development  of  Indices  of  simu¬ 
lated  battle  performance  should  Include  control  or  systematic  manipula¬ 
tions  of  the  above  variables  along  with  controller  Influence  and  terrain 
factors.  Indices  should  Include  additional  components  of  mission 
accomplishment  to  supplement  the  simulation  outcomes  used  in  the  current 
research. 
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APPENDIX  A 


Determination  of  Initial  Strength 

Initial  levels  of  OPFOR  were  determined  by  summing  the  assets  of  all 
first  echelon  maneuver  units  and  available  artillery,  regardless  of 
mission  and  whether  or  not  they  actually  became  engaged  in  battle.  Air 
assets  and  air  defense  weapons  were  not  Included,  since  they  had  no 
combat  effectiveness  weights  in  the  CATTS  model.  However,  both  OPFOR 
and  friendly  aircraft  inflicted  casualties  that  were  Included  in  total 
losses. 

Initial  levels  of  blue  strength  Included  the  maneuver  strength  of 
units  under  the  direct  control  of  BCG,  plus  adjacent  unit  strength  when 
it  was  engaged  in  battle.  In  all  scenarios,  BCG  units  performed  attack 
and  defense  missions  in  conjunction  with  adjacent,  brigade  controlled 
battalions.  For  example.  Inf  1  and  Inf  8  performed  a  Sinai  static 
defense  while  adjacent  battalions  performed  a  covering  force  mission. 

In  attack  scenarios,  BCG  controlled  forces  were  also  assisted  to 
differing  degrees  by  adjacent  battalions.  In  Cav  attack  scenarios,  Cav 
units  performed  a  screening  mission  for  adjacent  units  which  occasionally 
became  engaged  with  the  OPFOR.  In  all  the  above  situations,  adjacent 
units  received  and  inflicted  casualties  on  the  OPFOR.  Since  the  Impact 
of  adjacent  units  was  potentially  significant  in  the  above  scenarios, 
the  strength  levels  of  these  adjacent  units  were  added  to  BCG  controlled 
unit  strength,  and  casualties  inflicted  on  and  by  the  OPFOR  were  included 
in  total  loss  figures.  Artillery  and  air  strength  were  not  included  in 
initial  strength  levels  for  friendly  units  because  these  assets  were  not 
under  the  direct  control  of  BCGs. 

The  fact  that  a  majority  of  a  motorized  rifle  regiment  was  included 
in  the  initial  strengths  of  OPFOR  when  friendly  forces  were  attacking  or 
defending,  had  the  effect  of  reducing  simulation  outcome  values  when 
friendly  forces  were  attacking.  This  caused  the  distributions  of 
simulation  outcomes  for  defense  and  attack  missions  to  overlap.  This 
manipulation  allowed  a  comparison  between  controller  ratings  and 
simulation  outcomes  across  mission  types. 
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APPENDIX  B 

Calculations  of  Simulation  Outcomes* 


Loss  Exchange  Ratio  - 


Total  OPFOR  Losses 
Total  Friendly  Losses 


Relative  Exchange  Ratio 


Percentage  of  OPFOR  Lost 
Percentage  of  Friendly  Forces  Lost 


Surviving  Maneuver  Force 
Ratio  Differential 


Percentage  of  Friendly  Forces  Surviving 
minus  the  Percentage  of  OPFOR  Surviving 


C^ILL  Ratio 


1/2  (Percentage  of  Friendly  Forces  Surviving) 
plus  the  Percentage  of  OPFOR  Lost 


ACR  Combat  Ratio  -  Initial  Combat  Ratio  minus  Ending  Combat  Ratio 

Initial  Combat  Ratio 


All  losses  are  based  onZEW  X  ET  per  exercise  where  EW  =  equipment  weighting 
factor  and  ET  *  equipment  type. 
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